the investigator should either select the best and delete the rest or combine them into one measure. This strategy not only reduces the number of analyses, thus diminishing the risk of false positive results, but also diminishes the risk of false negative results, because combining multiple measures of the same signal frequently results in "tuning out" much of the noise and thus "tuning in" the signal, resulting in a combined measure that is more reliable.
Having many measurements and diagnostic assessments may compromise the quality control of the measurements. When there are only a few crucial measurements on which the success of a trial depends, the investigator can spend a great deal of time and effort to select the best instruments, provide adequate training and orientation to the assessors, and institute adequate quality control procedures. But if hundreds of variables are collected, expedient measures of limited validity become a temptation, and the consistency and care in assessing each variable may be compromised. The fatigue of both participants and assessors can further impair the quality of measurements. What is sometimes called a "rich" data set that contains a large number of variables may, on closer inspection, be rich only in noise, not in signal.
Very large data sets require more staff effort to maintain and analyze. A large number of variables will not be helpful if key variables that will reflect the hypotheses are inadequately measured. Investment in quality control of key variables that includes error checking, detection, and correction procedures is critical to achieve a valid result.
How to Ensure Reliability and Validity of Measures
Reliability "refers to the degree to which the results obtained by a measurement procedure can be replicated," and validity "is an expression of the degree to which a measurement measures what it purports to measure" (Last, 1988). Seldom do diagnostic procedures in any area of medicine have a reliability coefficient above 80 percent. Many diagnostic procedures in common use have reliability coefficients between 40 and 60 percent. The issue of reliability of diagnosis in psychiatry has certainly received far more attention than has reliability of diagnosis in most other fields of medicine. But the principle remains: unreliability tends to attenuate power, necessitating larger sample sizes (Kraemer, 1979); therefore, it is especially important to develop reliable methods.
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